
Command Line Projects and the Unix Philosophy
Week 1
Topics:
- What are “data skills?” | Reproducibility and open science | How to learn bioinformatics | Documentation | The importance of caution
Assignments:
- Read through BDS Chapter 1… twice, and carefully
- Find and explore the supplemental materials for the chapter on GitHub
- Assignment 1 - Reflection piece on why you want to learn command line skills and best practices
- Set up your computer environment (Command-line, Git)
Resources
For your consideration:
- “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.” –Brian Kernighan
- “Since the computer is a sharp enough tool to be really useful, you can cut yourself on it.” – John Tukey
Back to top of page
Week 2
Proper Project Organization
Topics:
- One directory per project | data as ‘read-only’ | rules for naming things | project structure | documentation
Assignments:
- Read through BDS Chapter 2 at least once
- Work through BDS Chapter 2, following along in your own terminal (data used is in online repository)
- Assignment 2 - Create oganized project template using code
Resources
Practice
- Re-create your project directory template by copy-pasting each line of code from your assignment to make sure it gives the same result
- Spend time making sure that you intuitively understand relative filepaths and get comfy with the terminal
- Spend 2-3 hours mucking about in your terminal reworking the lines from Chapter 2 over and over until it feels normal
Back to top of page
Unix refresher and sequence data types
Week 3
Topics:
Assignments:
- Work through BDS Chapter 3
- Assignment 3 - Use pipes and redirects
Resources
Practice
Back to top of page
Week 4
Working with Sequence Data
Topics
Assignments:
- Work through BDS Chapter 10
- Assignment 4 - Trim reads, Count nucleotides, convert from fastq to fasta
Resources
Practice
Back to top of page
Week 5
Combining Unix Skills and Command-Line Software
Topics:
Assignments:
- Case study 1 - Run ITSxpress on fungal data
- keep complex results
- customize output for question at hand
- make it reproducible
- do it on a hundred files
- store all log data into one file - for how many sequences TOTAL were no ITS stop or start sites identified?
- push “workflow.txt” (not data)
- uses chapters: 2,3,10 (for+loop, grep, redirect 2>, flags)
Resources
Practice
Back to top of page
Week 6
Topics:
Assignments:
- Work through BDS Chapter 7
Resources
Practice
Back to top of page
Week 7
Topics:
Assignments:
- Continue working through BDS Chapter 7
- Assignment 5 - build tabular file from fasta database
Resources
Practice
Back to top of page
Finding and Retrieving Data
Week 8
Online Repositories and Approaches to Downloading
Topics:
Assignments:
Resources
Practice
Back to top of page
Working with Supercomputers
Week 9
Interfacing with Remote Machines
Topics:
Assignments:
- Work through BDS Chapter 4 before class this week
- Assignment 7 - build 3 separate SLURM scripts to run fasta analyses
Resources
Practice
Back to top of page
Week 10
Interfacing with Remote Machines, Continued
Topics:
Assignments:
Resources
Practice
Back to top of page
Version Control and Collaborations
Week 11
Git for Scientists
Topics:
- Git workflow
- GitHub
- Collaborating with Git
Assignments:
- Work through BDS Chapter 5
- Assignment 8 - Git collaboration and merge
- Group effort: Everyone (in turn) make changes to this repository
Resources
Practice
Back to top of page
Week 12
Topics:
Assignments:
- Work through BDS Chapter 12
- Assignment 9 - Git collaboration and merge-
Resources
Practice
- In-class collaborative name list
Back to top of page
Putting it all together
Week 13
Composing Full Pipelines
Topics:
Assignments:
- Continue working through BDS Chapter 12
Resources
Practice
Back to top of page
Week 14
Running a Pipeline on a Remote Machine
Topics:
Assignments:
Resources
Practice
Back to top of page
Week 15
Topics:
- Testing with toy examples
Assignments:
Back to top of page
Week 16
Where to go from here?
Topics:
Assignments:
- Assignment 10 - Reflection piece on what you’ve learned and what next steps you’ll take
Back to top of page